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Adding the phrase "Learning by Doing" to the title "The Field Experience in 
Research Training," would make more explicit the implicit theme of this symposium. It is 
one thing to analyze and critigue research studies, — guite another to carry one out. But 
the latter prepares for the former. If the hierarchy of our Taxonomy of Educational 
Objectives is correct, in practicing the skill of synthesis, one also practices the skill of 
analysis (there is evidence that analysis and synthesis are related hierarchically, Ayers, 
1966 and Kropp and Stoker, 1966). Thus, critiguing articles, the common exercise of 
research courses, economically accomplishes analysis and synthesis simultaneously 
through having students create and do research studies. Such individuals would probably 
be good at critiguing articles without even having explicitly practiced this skill. 

But anyone who has reguired such an experience in a research course and has 
watched students struggle through the various stages of the research process knows its 
time consuming nature. For the student it can be severely taxing. For the instructor, a 
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large course can be overwhelming. A greater variety of research can be encountered and 
understood if critiquing is practiced directly; hence students are assigned critiques, often 
to the exclusion of doing research. 

GOALS OF THE FIELD RESEARCH EXPERIENCE 
But some skills are simply not learned through critiquing; encountering design 
decisions already made is quite different from having to make them. Just as the 
accomplished athlete makes a difficult routine look absurdly simple through its polished 
execution, so the choices of the skilled researcher seem so obvious and right that there 
doesn't seem much to it. Further, when, in critiquing, students find the author has gone 
astray, they are "Monday morning quarterbacking," claiming in hindsight that certain 
choices should have been obvious and preferred. By contrast, for the student choosing de 
novo among the myriad of design possibilities, tne "correct" choice may not be nearly so 
obvious. Note "correct" is in quotation marks; one person's "correct" design choice is not 
always another's. Indeed, design choice is an art as much as science and as one selects 
among the possibilities, one gains an appreciation of: 
o the many trade-offs that must be made, 
o the inadequacy of resources and the temporizing that results, 
o the optimization of many functions such as how much time and energy is to be 

spent on formulating and reformulating the problem, 
o the choice of the right balance between an emphasis on internal versus 
external validity, 

o the possible use of a familiar test or procedure in place of less developed but 
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possibly better ones to attain audience credibility, 
o the allocation of resources to provide the strongest design and the greatest 

and most appropriate information yield 
(Krathwohl, 1985). 

The detail of those decisions is lost to the reader ignorant of a study's 
developmental history. One can partially remedy this by using a book like Golden (1976) 
in which they are sometimes revealed. Golden provides sample articles for a variety of 
research methods, accompanying each with a chapter of comment by one of the article's 
authors. Many chapters give students a picture of decision-making in action. But the best 
way to appreciate the optimization process, and the trade-offs involved, is to experience 
research first-hand. 

A related reason for direct experience is to appreciate the difference between doing 
research and reporting it. Doing research, particularly in the exploratory mode, is a 
highly creative act which can be described mainly after it has been accomplished. One 
can talk about techniques that stimulate creativity, but rules to follow, defy us. In the 
words of Feyerabend (1975) "anything goes." 

The reporting of research, however, is a different story. Here the expectations of 
editors and audience constrain reporting to a logical chain of reasoning which leads to 
one's conclusion. The various parts of the chain are much of the substance of 
research courses, but experiencing the development of such a chain brings an appreciation 
of reality that reading alone does not develop. One realizes that there are collected but 
unreported data, Events are often reordered into an internally consistent report. The 
study may be twfsted entirely away from its original target by unanticipated findings. 
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Finally, experiencing research qives practice in that most difficult task for graduate 
students, dissertation problem formulation — "problem homesteading," to borrow a term 
from my colleague, Dr. Louis Heifitz. They spend inordinate time finding a topic and 
cutting it to size. Through my observations and interviews, I have found problem choice 
quite complicated. But interviews suqgest that early practice facilitates later success. 

So there are good reasons for giving a field experience. But what about the amount 
of time and effort on the part of both student and instructor such a task requires? 

THE FIELD EXPERIENCE PROVIDED IN OUR COURSE 
We have been wrestlinq with this problem in a two-semester research methods 
course which is required of all Ed.D. students and which is an option for Ph.D. candidates. 
My colleaque, Dr. Vincent Tinto has taught it with me the last several years and Dr. Peter 
Mosenthal joined us this year; I serve as Course Coordinator. We give two field 
experiences, one in qualitative methods and one in quantitative methods. Since I have had 
a hand mainly in the latter, I confine this discussion to it. 

In a two section course totalling fifty or more students, clearly one of our dilemmas 
was how to obtain the necessary student and faculty time for carrying out a study. A 
partial solution involves adding, once-a-week, a one-hour discussion section beyond the 
three-hour class session at no extra credit. Various faculty serve as consultants to these 
qroups, an experience they have generally found to be enjoyable. With no advance 
preparation except reading the class handouts., they can keep up with whst is going on in 
\he course. We have had little difficulty in recruiting them. 
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Student qroup members are determined almost solely by when they can meet. With 
busy schedules on all sides, this is a complex problem ev-i-rn with small group size — five to 
six students plus the faculty consultant. Note I call the faculty, "consultants 11 rather than 
"qroup leaders," because that is their role. After group members become acquainted, they 
chose a chair from among their number who will help them orqanize for the tasks. Group 
members come from all areas of education, a characteristic many find an initial problem, 
but most come to appreciate. 

THE UN-STRUCTED FIELD EXPERIENCE 

In contrast to having each student do a study independently, our initial notion was 
that the qroup could plan and jointly carry out a study of their choice and invention, 
dividing the responsibilities and combining their efforts. Thus each could take part of the 
literature survey and they cc -id trade xerox copies or abstracts and talk about them in 
the qroup sessions. Similarly, each could experience gathering data from five to ter. 
subjects. That would give them a sufficiently large body of data to meaningfully analyze. 

Our initial efforts were mixed. Some groups jelled beautifully and produced 
publishable studies. Others tried for so long to find a problem that the time required to 
operationalize it created crises situations resulting in unpleasant interactions. 

We asked the faculty consultants to ^rinq earlier closure to problem formulation. 
With the students really in charqe, we were only partially successful. We might have had 
better luck if our faculty consultants had worked with us over a period of years, but for 
several reasons we have chanqed them every year. Most obvious is the need to spread the 
load, but equally important is communicating to faculty the nature of the research course. 
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Required courses must be kept close to what the faculty want them to achieve. Involving 
a significant portion of the faculty as small group consultants gives them first-hand 
knowledge of the experiences students are havinq, the content and skills emphasized, and 
the students 1 difficulties. They can then give us valuable feedback from their perspective 
on how to improve. 

THE PARTIALLY STRUCTURED FIELD EXPERIENCE 

With enouqh support and assistance from faculty consultants we hoped to avoid 
removing the "problem homesteading" from the field experience, but it proved necessary. 
We did so realizing that another course requirement, development of an individual 
research proposal would give practice in uhis skill. For further practice, students 
presented their initial proposal ideas to the small groups for constructive comment 
(emphasis has been placed on avoiding discouraging comments in the early sessions). 
Through this experience, students see their problems played out in the work of the 
presenting student and they take part in developing, practicing, and learning the positive 
steps that can prove helpful in such situations. They also take their turn presenting. 

For still further practice and guidance, from the beginning of the course we 
encourage students to keep logs of their idea development and submit them for dialoque 
with the instructor. Those that do so usually report this is a valuable experience; some of 
the logs detail quite nicely how the problems develop. Further, learning to keep such a 
notebook is good research training; unfortunately it is a skill more often practiced in the 
natural sciences. The demands of the course are such, however, that few students find 
time to add this extra. 
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So we curtailed the problem homesteading givinq all students a partially structured 
common problem. It was a sample survey because of the student interest in this method 
and because it easily allows each student to experience gathering data. Our second trial 
of the experience worked much better than the first. Using established instruments, 
qrouDs build their problem around the topic of the questionnaire, modifyinq it as 
appropriate to their problem, pretesting it, gathering their data, and discussing methods 
of analysis. Each student individually analyzes and writes-up the data. The latter is to be 
not a blow by blow sequence of group actions, but a report of publishable quality. 

We point out to the students that this is not unrealistic problem development; 
problems can start anywhere in the chain of reasoning — with unusual subjects, a 
particular treatment, a useful instrument (the? law of the "hammer", Kaplan, 1964), and 
especially with a theory or hypothesis. In this instance, students are starting with an 
instrument, must construct the problem, and develop a desiqn consistent with it. Since 
Dr. Tinto's research has been in the area of college drop-outs, with input from us, he took 
responsibility for choosing the instrument and providinq sample references and some 
structure. The material they received is in the appendix. 

It was interesting to see how the students structured their "instrument-defined" 
problem. Most simply examined the relations of the suggested variables among run-of- 
the-mill freshmen. But groups exploited their freedom and looked for differences 
between foreign and native students, work-study and non-work-study students and 
commutinq and residential students. 

We insisted that they pre-test whatever they intend to use, since we intentionally 
built errors into the original questionnaire. If they didn't spot them, pre-testing would not 
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only find them but would also pin-point problems in items they may have added. 

RESULTS OF THE PARTIALLY STRUCTURED FIELD EXPERIENCE 

This successful exercise resulted in a large number of excellent papers. But, some 
groups still had difficulty focusing on a particular slant to the problem. The resulting 
time-press was, however, nowhere near as severe as formerly. Further, all groups found a 
reasonable problem formulation which did not always occur when the groups defined their 
own problems. 

Structuring seems not to have reduced learning from their mistakes, a source of 
concern to us. There are plenty of errors, including common ones. Following return of 
the papers, we devote class time to providing each group an opportunity to describe what 
they did and what they learned. 

DATA ANALYSIS 

One of the most difficult aspects has been to determine how much group analysis to 
encourage and how much to insist they do individually. Individual ingenuity shows at this 
staqe, some develoDing quite clever analysis techniques. But while we get creative 
graphics and comparisons, we ask for no complex statistics. Even in a two semester 
course, given their highly uneven background, we can teach little more than simple 
descriptive statistics — data displays, measures of central tendency and dispersion, simple 
correlation, and the iogic of inference (not how to do specific tests, just the logic). We 
emphasize getting close to the data rather than depending on the statistics even to the 
point of encouraging the use of such archaic schemes as edge marked cards (see sample in 
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the appendix). If they wish to learn computer routines, we ask they use those which give 
them scatterplots not just statistics. We were surprised at how many voluntarily learned 
computer packages and are very gingerly considering adding learning Supercalc3 or 
Minitab to an already jam packed course. Our biggest criticism is how much new material 
the course already attempts to cover. 

IMPORTANT ADDITIONAL GOALS OF THE FIELD RESEARCH EXPERIENCE 
The goals set in the first part of the paper for the research experience imply 
criteria by which the suitability of the experience may be judged and it seems clear how 
our class meets them. There are, however, two important additions: (1) Every problem set 
for the students should lead to the fun of discovery. This one did for most students and, 
given an interesting topic, that is not a hard criterion to meet. (2) This problem met a 
special criteria we have adopted as an essential characteristic of a class problem that the 
data contain rewards for students who examine it closely. In this day of computer 
packages that spit out a wealth of descriptive statistics, it is easy for students to depend 
on them to interpret the data. This year's problem was particularly rewarding if students 
studied the data not just the statistics. They became quite excited when some discovered 
that their scatterplots neatly showed contingent conditions. 

If we think about a cause as consisting of the necessary and sufficient conditions, 
then we can distinguish four sets of conditions: 
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Possible Combinations of Necessity and Sufficiency. 

Cause is sufficient for the effect 
Yes No 

Yes Only a single set A contingent 

of conditions condition 

Cause is necessary 
for the effect 

No An alternative set A contributing 

of conditions condition 

(For further explanation, see Krathwohl, 1985, pp. 218 & ff) 

Both continqent and contributing conditions hide behind low Pearson product-moment 
correlations. For example, the scatterplot from the data of Ms. Susan Allen (Figure 1) 
shows the relation between a social satisfaction index and the size of community of origin 
for freshmen students. In this scatterplot, dissatisfaction is contingent upon coming from 
a small community, but obviously satisfaction may also be gained as a result of other 
factors which apparently compensate for whatever coming from a large community gives 
one. The whole upper right corner, as indicated by the diagonal line, is devoid of cases. 
But they are scattered across the lower left-hand corner, a pattern typical of contingent 
relations. For this urban university, the larger the community of origin, the more likely 
the student has a high satisfaction index, but the reverse is not true. As the size of the 
community decreases, factors other than size apparently determine level of satisfaction, 
so it is unpredictable. 

We are careful to emphasize in class that such post hoc interpretations must be 
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cross validated. In addition we also note that the researcher is in 
a better position to know and interpret the data than anyone else. It is from this process 
that discoveries are often made. 

Contingent conditions will be found wherever a given condition must be met prior to 
a next condition. This is the case with all so-called "entry behaviors 11 in programmed 
instruction or mastery learning. Since such contingent conditions show up as triangular 
plots or as triangular plots set on top of a rectangular one, our conventional correlational 
statistics fail to describe the relationship adequately (Ibid., p. 221 <5e ff). Thus careful 
interpretation of the scatterplot is necessary. But when the relationship is discovered and 
understood, the result is 3 reinforcement of this kind of close data analysis. The presence 
of contingent relations, like this one, will be criterion to be met by the problems we set 
for future students. 

THE INDEX REQUIREMENT - THE ILLUMINATION OF ANOTHER RESEARCH ASPECT 
The index of social satisfaction in Figure 1 was developed by the student in response 
to the project instructions. We asked them to construct indices of student experiences 
and/or satisfactions. Although as indicated in the last paragraph of the second page of 
the instructions, this simplifies the problems of analysis, we had a second agenda. W e 
wanted students to encounter the problem of determining the explicit and implicit 
weightings of various items as they entered into the combination that constitutes an 
index. Havinq already given them some instruction and practice on conceptual analysis 
earlier in the course, this experience gave further experience as they defined the nature 
of "social satisfaction." Further they encountered the problem of operationalizing the 
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construct. Accustomed to accepting the composition of established tests, we wanted the 
students to face the judgments involved in choosing what kinds of items to include, how 
many, and how each should be weighted. More of them understood explicit weighting in 
terms of multiplying an item response by a weiqht than implicit weighting, for instance, in 
terms of the number of items measuring the same aspect of the construct, or in terms of 
its variance or its correlation with other items. The latter two may be beyond what we 
can expect, but not the first two. 

SUMMARY 

In summary, we have had some success in using a structured problem for the 
quantitative field experience. Some problems clearly provide better experiences and are 
more reinforcing of the students 1 efforts than others. The one we picked had rewards for 
those who became intimately familiar with the data rather than relying solely on 
statistics, a criterion we are inclined to think all such problems should meet. Pre- 
structuring the problem reduced - practice in "problem homesteading 11 which had to be 
picked up with other experiences. But pre-structuring the problem had advantages in 
reducing the burden of faculty support and resulted in a more universally satisfactory 
experience for the students. Using a survey type methodology allowed students to 
pxperienc^ data-qathering as well as the other facets of the process. Requiring indices 
introduced an understandinq of some measurement problems. Requiring individual 
analyses allowed freedom to show considerable ingenuity and requesting reports in journal 
format helped to delineate the process of doinq the study from how it is described. We 
h9ve no doubt we have considerable room for improvement, but we have msde 
considerable proqress from where we started. 
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EDU 787: RESEARCH CORE 



SURVEY EBCtfECT 

Attached you will find a survey questionnaire , In draft for*, 
which la Intended to ascertain to what degree end In what fashion 
freshmen year experience* ere related to the Intention to leave Syracuse 
University. But It Is not a complete nor entirely correct Instrument. 
Your task will be to complete and correct the Instrument, administer It 
to a population of students, end analyze the resulting data. To do so 
you will have to attend to the following tasks: 

1 . CQHPLETE_THE BACKGROUND INFORMATION SECTION. Three questions 

ere already written. Write et most seven <7) more questions whose 
purpose it is to ascertain the background attributes of the target 
population. Before you write the questions, you will* have to decide who 
the terget population will be end what pieces of information should be 
obtained about the characteristics of that papulation. Though there may 
be many pieces of Information you may wish to collect, not ell will be 
central to your concerns. For Instance, if you ere Interested in how 
the experiences of males and females or of traditional and adult 
students students ere related to the intention to leave, you will have 
to ask about the sex cad/or ege of the respondent. In doing so, you 
hope to be able to answer the question not only how experiences differ 
emong different students but elso how they may be differentially related 
to the intention to leave. 

2. COMPLETE EXPERIENCE ^D^SATISFACTION^^SECTIONS. Given your 

reading of the literature (see attached reference list), you mey decide 
to edd several additional questions to the second end third sections on 
student experiences end satisfactions. If end when you do so, please 
make sure you first consider why the requested information is necessary 
end in what fashion that information may be best obtained <e.g. 
open-ended or closed-ended form, verying types of scales, etc.). 

3- tlLOT.TEST^^RVEY^INCTRyMENI . The key to the success of eny 
survey Instrument is pilot testing. It enables the reseercher to check 
net only on possible errors either in form or in interpretation, but 
elso for ereas fox additional questioning. In this instence, the 
second (experiences) end third <»atiaf actions) sections of the 
questionnaire have several errors which need correcting. Furthermore, 
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they cover some, but Dot all, of the possible areas of inquiry on the 
character of student experiences. Your task Is to pilot test the full 
instrument and ascertain if any changes and/or additions need be made 
before it is administered. Here let the several persons you choose for 
pilot testing act as a snail panel of -experts** and advise you on how 
the instrument can be improved to better obtain the information needed 
to answer ti.e research question. 

«• 5^CT_TARGET_P0PULATig^ Decide on the 

target population and the procedures you will employ to reach that 
population. Mote that the decision on target population and the 
completion of the background section may be related in that certain 
choices in population and procedures (e.g. stratified sampling) may 
obviate the need for certain types of background information (e.g. you 
choose to survey females only). 

s - ¥BIII_EI?AL_yERSION_OF_SUR^ . Given the results of 

the prior tasks, you must finalize the questionnaire and write an 
introduction to it which helps the respondent understand both the 
purposes of the instruments and the conditions under which it will be 
used (e.g. confidentiality). 

6. AD5INISTEF_THE_™^ Administer t> 
instrument and compile the data. In this instance, you may choose 
have each member of the group obtain ten to fifteen separate 
questionnaires or allocate tasks such that one or two collect all the 
data which are then partitioned among the group for analysis. However 
collected, the data should be compile, coded and put into a form such 
that data analysis can be performed. 

7. AMALY2E_THE_DATA . Carry out simple univariate statistics to 
describe the data (e.g. means, standard deviations, etc.). Then perform 
a selected series of cross-tabulations (correlations are optional) in 
order to escertain how -axperiencee are, for different types of students, 
related to the intention to leave. As part of that analysis you may 
want to use the collected data on vocetional/social/acadamic orientation 
of students. Those data, frequently used elsewhere, provide insight 
into the orientations of students regarding the importance of different 
ereas of endeavor. Bore importantly, they sometimes prove to partially 
explain how experiences come to influence decisions to leave. 

To simplify the task of analysis, you should also attempt to 
construct an index of student experiences and/or student satisfactions. 
Generally speaking en index is a single measure which best captures a 
complex concept which is normally measured by a range o£ separate 
items. The task here is to decide how (end why) you will combine the 
separate measures of experience end/or satisfaction so as to produce e 
single measure which can then be used in the analysis. 
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8. WRITE A__PROJECT RESEARCH REPORT. Together with a brief 

literature review (taken fro* the appended reference list), write up e 
project research report. Though you »ay combine your data end work 
together on the data analysis and literature review, each person should 
write a separate report on the project* You say, however, ease the task 
by asking different aeabers of the group to read different references. 
The combined readings say then £>e used by each aevber of the group for 
the literature review of each report. 

Please note that your reports should contain a closing section on 
what things you would do differently were you to do the research again. 
In other words, you should speak to the question of how the research 
project could be improved the next tise it is carried out (please oait 
the possibility the reconsendation that the project never be carried out 
in the future! ) . 



Dr. Vincent Tinto 
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FFSSHMAN QUESTIONNAIRE 



BACKGROUND 



I am (please circle) Male Female 

What is your current BtatuB? New Student Transfer 

From which type of coBBunity do you coie? (Circle one) 

a. Rural farm or country d. Suburb of a city 

b. Small town (less than 10,000) e. City (lore than 50,000) 

c. Large town (10,000 to 50,000) 

What is the highest level of formal education completed by each 
of your parents? (Circle one in each column) 







Mother 


Father 


a. 


Grades 1-9 


(1) 


(1) 


b. 


Some high school 


(2) 


(2) 


c. 


High school graduate 


(3) 


(3) 


d. 


Some college 


(4) 


(4) 


e. 


Bachelor's degree 


(5) 


(5) 


f. 


Graduate degree 


(6) 


(6) 
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II. EDUCATIONAL EXPERIENCES 



For each of the following educational situations or activities, please 
indicate the extent to which you have participated in those activities 



during the lost year <1 = very little/none, 2 = 
Boat other firat year students, 4 = a great deal) 


some, 3 


= more 


than 




Amount of 


Participation 


a." Meet with faculty' out aide claaa 


<1> 


<2> 


<3) 


<4) 


b. Meet with ataff (e.g. advisor, counsellors) 


<1> 


<2> 


<3> 


<4) 


c. Diacuaaed serious topica with other atudenta 
outaide claaa 


<1> 


<2> 


<3) 


<4) 


d. Initiated an appointment with a faculty member 
or ataff member to gain needed information 


<1> 


<2> 


<3> 


<4) 


e. Attended a cultural activity sponsored by the 
university or a student group 


<1> 


<2> 


<3> 


<4) 


f. Uaed the library to gather aateriala and 
Information 


<1> 


<2> 


<3> 


<4> 


g. Participated in an extracurricular activity 


<1> 


<2> 


<3> 


<4) 


h. Attended special lectures or exhibits 


<1> 


<2> 


<3> 


<4) 
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III. EDUCATIONAL SATISFACTIONS 



Kow satisfied are you with your experience at Syracuse University thus far? 



For each of the following educational or social activity, indicate your 
level of satisfaction by circling the appropriate number. 



< 

4 


1 = very unsatisfied, 2 = somewhat unsatisfied , 
= very satisfied ). 


3 = 


somewhat 


aotiafied 




Level 


of Satisfaction 


a • 


Quality of teaching 


\ 1 - 


r U) 






b. 


Frequency and quality of contact with faculty 


<i: 


> <2> 


<3> 


<4> 


c. 


Frequency and quality of contact with staff 


<i: 


> <2> 


<3> 


<4) 


d. 


Quality of cultural life 


<i: 


l <2> 


<3> 


<4) 


e. 


Quality *of social life 


<13 


l <2> 


<3> 


<4) 


f . 


Quality of library 


<i: 


l <2> 


<3> 


<4) 


9- 


Quality of residence halls 


<i: 


> <2> 


<3> 


<4) 


h. 


Friendliness of students 


<i: 


l <2> 


<3> 


<4) 


i. 


Friendliness of faculty and staff 


<i: 


l <2> 


<3> 


<4> 




Intellectual progress thus far 




> <2> 


<3> 


<4) 
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IV. INTENTION TO LEAVE 



1. At this time, do you plan to return to Syracuae University next year? 

(a) Yes, definitely 

(b> Yea, maybe 

(c> No, maybe 

<d) No, definitely 



2. If you do not intend to return to Syracuae next year, could you pleaae 
indicate why. 



3. If you do not intend to return to Syracuae next year, what do you intend 
to do? (Circle as many as applied) . 

<a) Tranafer to another four-year institution. 

<b> Transfer to a two-year college. 

<c> Obtain employment* 

<d) Other (please explain) 
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Relationship of Size of Community to Satisfaction 
(measured by Satisfaction Index) 

Small Town Small Town Large Town Large Town 
Satisfaction Index Rural (not suburb) (suburb) (not suburb) (suburb) City 
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